1. The Field of the Invention
The field of the invention is the estimation of hardware and software resource needs for a server application given the anticipated number of users. More particularly, the field of the invention relates to methods for creating hardware and software resource usage models that can be used to estimate hardware and system resource usage in a wide range of different hypothetical usage scenarios.
2. Present State of the Art
Increasingly, client-server computing has become a normal trend for the delivery of computational services. A server computer will have a desired application and a client computer will have software capable of accessing the server computer for the desired services. This may take place over a communication network, such as an Ethernet Local Area Network ("LAN"), or it may occur over a more direct connection through a modem. In either case, the bulk of the computing is provided by the server computer or server application and the results are communicated to the client portion or client application that will then display the results for the user.
It becomes a somewhat difficult process in many instances to anticipate adequate resources for running the server application for a certain level of load created by a particular number of users desiring the services provided by the server application. The resources used are typically hardware resources, such as CPU time, network bandwidth, disk and memory usage, etc., but may also include software resources, such as operating system facility usage.
It is important to anticipate adequate resources for the server application in order to reduce frustration on the part of the user and to allow the client-server software to work most effectively. One of the most common user frustrations is having an unusually long delay in receiving the results of the services provided by the service application or simply not having the server application available to provide the desired services, both due to inadequate hardware and system resources. Furthermore, actual errors in the services provided may occur due to the lack of resources available for running the server application.
While such problems apply to many different server applications, throughout this patent application, the example of an e-mail server as part of an on-line service, such as Compuserve.RTM. or America Online (AOL), will be used for illustrative purposes. Such an illustrative context is not to be interpreted as limiting to the present invention since any server application can be used as part of creating an estimation model. Some other server applications that could be used and where it is important to accurately estimate the amount of necessary resources would include web page servers, network file servers, etc.
With an e-mail server as part of an on-line service, a user having client software will dial up and gain access through a communications network to the e-mail server in order to download received e-mail messages and upload messages for delivery to other users. One scenario of how an e-mail application server is accessed by an on-line service subscriber or user is as follows. The user will begin by preparing messages for delivery to others using the client application program provided by the on-line service on the client computer. After such messages are prepared, the user will log onto the on-line service and access the e-mail server application with the client application most likely through a direct modem connection to a locally available communications network that allows communication between the client application and the server application. With the connection thus made, the client application will upload the messages to the e-mail server application for delivery. Finally, the e-mail server application will deliver the messages either locally to another on-line service subscriber account or remotely across the Internet or other communication network to a non-local destination.
The general scenario for reading new messages delivered to an on-line service subscriber or user includes first logging onto the on-line service in order to make the logical connection through the communications network between the client application and the e-mail server application. Next, any messages received for a particular user are downloaded to the client application so that the connection need not be maintained indefinitely. Typically, once the messages are downloaded, the logical connection between the client application and the e-mail server application is terminated. Finally, the user may use and peruse the messages leisurely on the client computer using the client application. These processes of sending and receiving messages will be explained in more detail hereafter.
A subscription service may reasonably and accurately estimate the number of anticipated users and the user behavior in terms of the number of logons per day and messages sent per logon and other relevant metrics by monitoring current subscriber behavior or by other rational means. It is often very difficult to predict, however, how much hardware resources would need to be dedicated to an e-mail server application in order to adequately service the anticipated user demand. The different amount of resources that an e-mail server application (or any other application) would use include CPU time for executing server application code to handle the logons, downloads, message sends, etc, network bandwidth for communication between the client and server for each download/upload or logon, memory necessary to perform the relevant operations, and the disk storage necessary for storing the received messages per user that would fluctuate with the actual user activity.
One can readily see that creating an accurate model that is scalable for different numbers of anticipated users can be a very difficult task due to the many variables involved and the nature of the server application selected. It is nonetheless important to make as accurate estimation as possible the amount of hardware resources required for adequate service.
One way of attempting to accurately gauge the amount of hardware resources necessary for supporting a particular server application is through the use of relevant "benchmark" tables. One or more hypothetical hardware configurations running the designated server application are set up and performance tested with a load created by a number of actual or simulated users. Relevant data are measured such as user response time, total CPU usage, disc usage, etc. so that each benchmark configuration is "rated" to support so many users. A potential user or marketing representative selling the server application may then utilize the tested configuration to estimate what may be the actual system purchaser's needs.
The benchmarking method, however, is fraught with potential error and amounts to little more than educated guess work unless the user's anticipated needs are very near the loads created for the bench mark configuration. For example, the benchmark user behavior may be markedly different than the system purchaser's user behavior rendering the configuration "rating" information essentially meaningless. Furthermore, if the purchaser's anticipated number of users is dramatically different than the "rated" number of users for the benchmark system, then a system purchaser could easily over or under purchase hardware for their particular needs.
The accuracy of bench marking may be increased somewhat by utilizing a series of three or more configurations from which to gather data. For example, there could be a low, medium, and high level of users with corresponding hardware configurations that would support the server application at a desirable performance level. The drawback for this approach is that setting up and running tests for different configurations can be expensive in terms of time and capital expenditures for the relevant hardware. Furthermore, the number of different configurations that need to be tested can be quite extensive. Such a number of configurations is exponentially increased when testing is done in combination with other server applications that will also be utilizing the same hardware resources.
In summary, benchmarking can be done in two basic approaches. The first approach picks a popular hardware platform having a fixed set of resources. The typical user behavior for the server application is then determined by reasonable guess work or empirical testing. Next, tests are run to see how many users doing the specific behavior and interacting with the chosen hardware configuration can reasonably support within certain performance characteristics. This "capacity" or "rating" is then published for the given hardware. The second approach uses tests run on multiple hardware configurations and determines the "capacity" based on a price performance curve across the capacity numbers of the various configurations.
In either approach, a number of concurrent users with a specific behavior on a specific set of hardware is used, and the purchaser is forced to either guess or extrapolate if their situation is anything different than the benchmarked systems in terms of either user behavior or existing hardware that may be used. Sometimes there is really no solid information as to the user behavior used in the benchmark. In many instances, buying new hardware is not always a viable solution due to the sizable investment previously made in existing hardware and in no case do the combined effects of different server applications exhibit themselves in benchmark tests of individual server applications.
It would generally be desirable to abstract a model so that resource requirements to run a particular application could be accurately estimated given anticipated users. Additionally, it would be desirable for the resource usage model to accommodate customized user behavior to enhance the accuracy of resource usage estimates. Furthermore, effective and accurate ways of developing such models that can be readily implemented into a computer are also sought. What is needed is a way of quickly developing an accurate model for hardware and software resource utilization that can be accurately scalable across a wide spectrum and can be used by server application sales representatives to accurately assist a customer in providing adequate hardware resources to accommodate the anticipated load.